Structure and Motion in Man-made Environments

3-dimensional (3D) computer graphics are rapidly becoming a crucial tool in such diverse application areas as design, the film industry and the world wide web. Because of this there is great demand for realistic 3D models of real world locations. Producing such models manually is a painstaking process, and so the ability to generate realistic 3D models automatically from a collection of images of a scene has become hotly pursued within the field of computer vision. In recent years many approaches to this problem have been tested, but no general solution has been found. Some systems extract point features from each image automatically and try to determine the position of each point in space, but this has proved to be difficult to achieve reliably. Other systems are interactive, having a user specify some information about the model and attempting to infer structure based on geometric and image based constraints. However the usefulness of these systems is generally limited by the patience of the user, as the detail of reconstruction depends on the detail and accuracy of information provided. An alternative approach, proposed by this thesis, is to have the user initialise a very coarse structure estimate, and then to refine this estimate using information extracted from images of the scene. The algorithm described in this work sequentially refines an approximately planar scene, representing its structure as a mixture of bas-relief and planar regions, while also recovering the motion of the camera. The results presented confirm the promise of this approach for generating realistic, accurate models from very few uncalibrated images. The work documented in this thesis is my own, except where otherwise noted. The majority of the original work in this thesis is contained in the model refinement algorithm described in Chapter 3. I would like to thank Dr. Roberto Cipolla for his guidance and support. I am also grateful to Dr. Tom Drummond for his many helpful suggestions, and the Cambridge Commonwealth Trust for providing the financial support to pursue this work. Anthony Dick